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A review of the literature indicates that, compared 
to larger, urban schools, small, rural secondary schools have limited 
course offerings in mathematics and science; the review also 
indicates, however, that the achievement of students in small, r iral 
schools is equivalent to or even higher than, that of students in 
larger, urban schools, such an anomaly may be due to a mismatch of 
the conventional measurement of achievement outcomes and the neglect 
Of higher-order cognitive skills taught in advanced courses offered 
only in larger schools. This study examines another possible 
explanation for the anomaly by focusing on the measures of 
higher-order thinking skills in science and mathematics. The data 
used in this study wert collected beginning in 1987 by the 
Longitudinal Study of American Youth (LSAY) , an on-going 4-year panel 
study of middle an<^ high school science and mathematics education. 
The base-year sample consisted of 2,829 loth grade students from 51 
randomly selected public schools. Survey instruments were completed 
by the sampled students, their teachers, and parents. Achievement 
tests that focused on mathematics and science knowledge and 
higher-order thinking skills were administered in the fall of 1987, 
1988, and 1989. Zero-order statistics and regression analyses were 
performed with the data. Results of the study refute the hypothesis 
and indicate that while large schools offer more advanced courses 
than do small ones, those offerings appear to have no influence on 
student achievement and higher-order thinking skills. (LP) 
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Small Schools and Higher Order Thinking Skills 

Introduction 

In the social science s theories frequently collide with facts. That is 
hardly news, of course, out the collisions are interesting; they incite 
speculation and inquiry. Seemingly, such an anomaly has occurred in studies 
of school size and student achievement. 

Consider. On the one hand, there are good theoretical grounds for 
believing that small (usually rural) high schools ^ dll not offer as 
comprehensive programs as larger (usually urban) ones. The notion of 
economies of scale compels us, a priori, to such a conclusion. Further, this 
conclusion has ample empirical support; small schools offer fewc;: courses and 
less comprehensive programs than do larger ones (Barker, 1985; Monk, 1987; 
Haller, Monk, Spotted Bear, Griffith and Moss, 1990). It would seem to 
follow, then, that student achievement would be lower in small rural schools. 
After all, when only two years of mathematics is offered by a high school, all 
else equal, students will presumably learn less math than if they were 
attending institutions that offered four yeai-s of that subject. 

On the other hand, there are the facts. Many studies show that the 
achievement of students in small and rural schools is as high as that of 
students in larger, more urban institutions (e.g., Kiesling, 1968; Edington & 
Martellano, 1984; Summers & Wolfe, 1977). Indeed, by some accounts their 
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achievement is higher (e.g. Fowler & Walberg, 1991). Thus, the puzzle: How 
can students who are able to study only one or two years of a subject learn as 
much (perhaps more) than students who are able to study four? 

In this paper we examine one possible explanation for this apparent 
anomaly: Perhaps the usual measures of achievement are inappropriate for 
the comparisons being made. We suggest that instead of reljdng on common 
standardized achievement tests to measure learning outcomes, researchers 
might better focus on competencies that are the likely outcomes of advanced 
study in a curricular area. We have in mind such competencies as the ability 
to synthesize information, to evaluate ifacts and procedures, and to apply 
abstract concepts to practical problems. Schools that are able to offer a rich 
variety of courses in a subject area may have little advantage over more 
impoverished ones in so far as basic skills are concerned, but might have 
large advantages when higher-order cognitive skills are the criterion. In 
short, the theoiy may be right, but we may have been looking at the wrong 
facts. 

Background 

According to traditional arguments, small schools are inferior to their 
larger counterparts (see e.g., Knezevich, 1984). These arguments have not 
been mere academic exercises. They have had the very practical consequence 
of causing the disappearance of tens of thousands of schools and school 
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districts during this century (Pipho, 1987). Every state has sought to 
improve its educational system by policies that either promote or require 
school consolidation. 

These arguments are particularly compelling when we consider 
curriculum adequacy. There are now numerous studies of the effect of school 
size on curriculum offerings. Several authors have shown that large 
secondary schools are able to offer a greater number of courses than small 
ones (Barker, 1985; McKenzie, 1989; Melnick, Shibles & Gable, 1987; Monk, 
1987; West, Miller & Diodata, 1985). Further, it is not simply a matter of 
course numbers. Large schools typically offer more comprehensive programs 
than do smaller ones, and especially relevant to the case at hand, this 
greater comprehensiveness is evident in science and mathematics. While 
curricular comprehensiveness varies by subject area, in no area, whether 
academic or vocational, are the smallest schools the curricular equals of the 
largest (Haller et al, 1990; Monk & Haller, 1991). 

Of particular relevance to this study is the finding that most high 
schools of whatever size offer a basic introductory course in each subject 
area. In mathematics, for example, most high schools offer a course in 
Algebra. As they become larger, more advanced courses are added, e.g.. 
Analytic Geometry, as well as courses targeted at either talented students, 
e.g., Honors Algebra, or those needing remedial work, e.g.. Consumer 



MaUiematics (Haller et aL . 1990; Monk & Haller, 1991). Thus, comparing 
students from small and large schools on measures of basic concepts and 
skills taught in introductoxy courses, especially when those courses ax'e 
required by states (as they often are), is unlikely to turn up either substantively or 
statistically significant findings.^ Rather, it would seem necessary to seek 
differences in those skills that are the products of courses offered by large but not 
small schools. 

School size may not be the only factor of importance. There is research to 
suggest that a school's location can add significant constraints to its curricular 
offerings. Specifically, location in a rural area may further hinder the capacity to 
offer curricula that are comparable in quality to those offered in more urban 
places. Such constraints stem from several sources. For example, rural schools 
have difficulty in recruiting and retaining teachers with credentials that are 
comparable to those found in suburban and urban areas. In particular, Carlsen 
and Monk (1992) found that secondary science teachers in rural schools were less 
likely to have majored and science as undergraduates and reported taking fewer 
science and mathematics courses than their counterparts in suburban and urban 
schools. These teachers may also have less adequate equipment and facilities. 
The long distances that pupils must travel to school and to area educational and 



^ In this regard it is important to note that the relationship between school 
size and course offerings appears to be curvilinear, where the degree of the 
departure from linearity varies depending on the subject area (Monk & Haller, 
1991). 
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technical centers present serious scheduling constraints on students' programs 
(Monk & Haller, 1986). Finally, lower aspirational levels on the part of students 
and parents (Cobb, R. A., Mclntyre, W. G. & Pratt, P. A., 1989) may lower the 
probability that rural schools will oSfer advanced, academically-oriented courses, or 
tliat students will take them when they are offered. 

It would seem hard to escape the conclusion tliat in regard to curricula at 
least, small rural secondary schools do not provide equal educational opportimities 
to their students. If one asks "So what?" however, the issue becomes more 
complicated. Presumably the m^jor reason wliy curricular disparities are 
important is because they have important consequences for student achievement 
and, ultimately, for success in life. Indeed, if such consequences do not exist, it is 
not obvious why anyone should care. But when we examine the relation between 
the size of educational organizations and student learning, the expected results 
are conspicuously absent. 

There have been numerous studies of the relationship between school size 
and student achievement. With few exceptions, these studies seem to show that 
students in small schools are at least as successful as their peers in larger 
institutions. For example, Eberts, Kehoe and Stone (1984), Edington and 
Martellaro (1989), Fowler and Walberg (1991), Guthrie (1979), and Mehiick, 
Shibles and Gable (1987) suggest that either school size has no effect on 
achievement or that its effect is slightly negative. A modest exception to this 
evidence was reported in New York State, where Monk & Haller (1986) found that 
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in the vexy smallest high schools (i.e., those that enroll fewer than 200 students) 
achievement in certain subjects was slightly below the state mean. As another 
modest exception, Friedkin & Necochea (1988) found that the size-achievement 
relationship is a contingent one, i.e., that it is negative in low SES schools and 
positive in high SES schools. Even in the latter case, however, the positive efifect 
of size on achievement is quite small, while in the low SES case its negative effect 
is substantial. Overall, it seems safe to conclude that small school size does not 
lead to noticeable decrements in pupil achievement. 

A similar conclusion can be reached in regard to rurality: rural students are 
at no obvious disadvantage in so far as their achievement is concerned. For 
example, while Downey (1980) found that rural students in Kansas scored slightly 
below their urban counterparts on the ACT, he concluded that the difference was 
so small as to be negligible. In contrast, Easton (1985), Edington & Martellaro 
(1984), Mclntire & Marion (1989) and Ward & Murray (1986), foimd no 
significant differences in achievement between rural and urban youth. Examining 
university GPAs, Hand & Prather (1990) found that students firom rural high 
schools generally outperformed their urban counterparts when SAT scores and 
SES were controlled. Presumably, having experienced the restricted curricula 
rural schools offer, ceteris paribus, their students would be at a disadvantage in 
university courses. 

As we noted m the inti^oduction to this paper, these conclusions-that small, 
rural secondary schools have sharply restricted curricular offerings and that their 
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instructional programs are qualitatively inferior, while the achievement of their 
students is equivalent to that of students in larger more urban schools-present an 
anomaly. One possible resolution of this oddity is that researchers have relied on 
inappropriate measures of student achievement. That is, most of the research 
comparing the achievement of students in different size schools has relied on 
either of two kinds of achievement tests, neither of which is very suitable to the 
task. 

The first and most commonly used measures of achievement are the 
nationally standardized achievement batteries. The problem here is that these 
tests are not curriculum-specific. At best they test a sort of generalized knowledge 
presumably taught in all schools. Hence, they are imable to detect knowledge 
differences among students who vary in the amoimt of a subject they have 
studied. 

Alternatively, when curriculimi specific tests (such as New York's "Regents' 
Examinations") have been used to compare achievement levels in small and large 
schools, another sort of difficulty arises. The problem here is that these tests are 
typically focused on measuring the recall of simple facts garnered during the year 
or two that students are enrolled in state mandated courses. But virtually every 
school-large or small, rural or urban-offers its state's mandated courses. It 
would be surprising, for example, if students in large and small schools differ in 
their knowledge of the simple facts of American history. Virtually every pviblic 
school offers at least one course in that subject; thus, equivalent numbers of 
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students will know what the Bill of Rights is, regardless of the size of tlieir high 
school. However, only large schools are likely to offer a social studies course 
devoted to the U.S. Constitution. Hence, only their students are likely to 
understand tlie tension between the Establishment and Free Exercise Clauses of 
the First Amendment. 

Thus, it is possible that students in small rural schools are negatively 
affected by their schools' restricted curricula, but researchers haven't detected 
those effects because they have been using the wrong measures of achievement. 
In short, the appropriate tests should be bot * . ^^rriculum-spedfic and should 
measiu'e the kinds of outcomes that are more i kely to be the product of extended 
study in a subject, i.e., from taking the «^orts of courses offered by larger schools. 

Further, the subject matter areas chosen to search for a school-size effect 
must be selected carefully. As we noted above, small schools are more nearly the 
equals of larger ones in some subjects than in others. For example, the social 
studies programs of small and large schools are more nearly comparable than are 
mathematics programs, which, in turn, are more nearly equivalent than are 
science programs (Hallt^r ||||., 1990; Monk & Haller, 1991). 

In this paper w^ focus on measures of higher order thinking skills in science 
and mathematics. We suggest that extended, in-depth study, such as is possible 
in large schools that offer many advanced courses in those fields, is likely to result 
in students having the abilities foui^d at the upper end of Bloom's taxonomy of 
cognitive skills (Bloom, 1956). Thus, students attending larger, urban high 
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schools, ceteris paribus^ should be better able to apply concepts to real-life 
problems, to relate science and mathematical concepts to those from other 
disciplines, and to make valid inferences to novel situations. 

We would not argue that higher order thinking skills are taught only in 
advanced courses, although it is plausible to suggest that they receive more 
emphasis there. Clearly, for example, a teacher of Algebra I might emphasize 
such skills while a teacher of calculus might accent rote learning, giving no 
attention to synthesis, evaluation or the application of abstract ideas to practical 
problems. Instead, we would argue that tlie development of these skills requires 
practice, and that the more work students have in a particular discipline, the 
more able they will be to demonstrate these sorts of educational outcomes. Thus 
we hypothesize that the mean achievement level on tests of higher order thinking 
skills will be greater in large schools than in small ones. 

While it is the case that larger urban schools, on average, offer more 
advanced courses than do smaller rural ones, the relation between size or location 
and curriculum depth is far from perfect. There are small rural schools with rich 
mathematics and science curricula and large urban schools that are relatively 
impoverished in this regard (Haller, etal., 1990; Monk & Haller, 1991). Similarly, 
equal proportions of students may not avail themselves of the opportunity to take 
whatever advanced courses their schools offer. Thus, a relationship between size 
and rurality and higher order skills will depend on the presence of advanced 
courses and student course taking patterns. Finally, any relation between school 
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size and these skills may depend on the socioeconomic level of the school's 
clientele. Friedkin & Necochea's (1988) finding that size and SES interact in 
affecting the more usual measures of academic achievement may also hold in 
regard to high order skills. 

Based on these considerations, we hypot'iesizp that there will be a positive 
zero-order relation between high school size and location in an urban or suburban 
area and the average level of achievement of students on tests of higher order 
thinking skills. This relation may depend on the SES of the schools' cUentele. In 
either case, we hypothesize that when we control for the number of advanced 
courses schools offer and for the rate at which students enroll in those courses, the 
zero-order relations between size and higher order skills and between rurality and 
higher order skills will be reduced or eliminated. 

Methods 

The data used in this study were collected for the National Science 
Foundation under a contract with Northern Illinois University. The project, 
known as the Longitudinal Study of American Youth (LSAY), is a four-year panel 
study of middle and high school science and mathematics education. The 
collection began in the fall of 1987 and is ongoing. The results we report here are 
based on the original collection plus the first and second follow-up studies, 
induciing information collected in the spring of 1989. 
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The base year sample consisted of 2,829 students who were enrolled in the 
10th grade in the &11 of 1987.^ These students were drawn from 51 schools 
around the nation. The sample consisted entirely of public schools, which were 
selected randomly with probabilities proportional to their enrollment size within 
twelve sampling strata. These strata were defined in terms of geographic region 
(four categories) and community type (urban, suburban, rural). Sixty 10th grade 
students were randomly selected from within each school. When fewer than 60 
students were enrolled in the school's 10th grade, all students were drawn into the 
sample. 

Survey instruments were completed by the sampled students, their 
teachers, and their parents. In addition, achievement tests that focused on 
mathematics and science knowledge were administered in the fall of 1987, 1988 
and 1989. These tests utilized items developed by the National Assessment of 
Educational Progress (NAEP). The NAEP items permit an assessment of 
students' capabilities along several dimensions of achievement, including certain 
higher order cognitive skills. 

In science, the measure of higher order skills pertained to the integration of 
science knowledge (SCIENCE), This involved the student's ability to "organize 
information more generally for problem solving, often involving use of scientific 
concepts and piindples, drawing proper conclusions, and employing higher-order 



* The base year sample also includes 3,116 students who were in the 7th 
grade in 1987, but these data were not used in this study. 
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thinking skills in reaching a solution to a problem" (Miller, Siichner, Hoffer & 
Brown, 1990). In matliematics, the measure of higher order skills was also 
focused on problem solving. This scale (MATH) measured a student's ability to 
interpret underlying concepts, assumptions, and relationships and their use in 
solving non-routine, often multi-step problems (Miller, et al., 1990).^ Tests 
administered during the Fall of 1989, when students were in the twelfth grade, 
provided the data for our measures. The within-school means of both achievement 
scores were calculated and taken as the dependent variables of this study. 

We took as a measure of a school's size {SIZE) its 10th grade enroUment as 
reported by a school administrator. The socioeconomic level of the school (SES) 
was indexed by taking the average of the educational level of each student's 
parents and calciilating the mean of this for each school. Our measure of a 
school's location was based on administrators* responses to a question asking them 
to characterize their schools as urban, suburban or rural. These responses were 
coded as a dummy variable with urban and suburban as the reference category. 

Our measure of a school's course offerings as well as of students' course- 
taking patterns were based on teachers' responses to questions concerning the 
classes taken by LSAY students. In each semester the math and science courses 



^ The scores we used were estimated by Item Response Theory methods 
which recognizes that some items are more difficult than others. Students' 
response patterns were adjusted in light of these variable difficulty levels. 
The reliability coefficient for the science integration scale is .59; for 
mathematics problem solving, .84. 
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taken by each student were identified. If any of the sampled students in a school 
was reported as having taken, while a sophomore, junior or senior, an honors 
math course (e.g., "Honors Geometxy"), or a course in geometry, algebra 2, 
trigonometry, analytic geometry, pre-calculus, calculus or probability/statistics, the 
school was credited with offering tliat course. The number of unique such courses 
constituted our measure of a schools' course offerings in advanced mathematics 
(CRSOFFCM)). A similar procediu'e was followed for science course offerings: all 
honors coiirses, second year biology, anatomy/physiology, chemistry and physics 
were counted as advanced science course offerings {CRSOFF(S)).* 

The extent to which students took the advanced courses offered by a school 
was computed by first counting the number of these advanced math and science 
courses taken by each student. Then, the mean of each of these numbers was 
then calculated for each school {CRSTAKE(M)) and (CRSTAKE(S)). 

We used ordinary least squares regression analyses to examine the effects of 
size and rurality on higher-order thinking skills. In order to test the hypothesis 
that higher order thinking skills are positively related to school size and ruraUty, 
we regressed SCIENCE and MATH separately on each of those variables, with 
SES and a product term for the interaction of SIZE and SES in the equation. 
Next, to test the idea that the effect of school size and rurality on higher order 
skills is due to the ability of large urban schools to offer advanced courses in 



* See Haller et al. (1990) for a full discussion of the rationale for this 
coding scheme. 
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science and mathematics, we entered CRSOFF(M) and CRSTAKE(M) or 
CRSOFF(S) and CRSTAKE(S) into the appropriate equation. If we are correct, we 
should expect to see the regression weights for school size and rurality decrease 
significantly in magnitude. 

Results 

Table 1 provides descriptive information for the variables used in this study 
as well as a zero-order correlation matrix. With listwise deletion of missing cases 
in effect, 47 schools remained for analysis. It is obvious from Table 1 that, imless 
there is a substantial interaction between school size and socioeconomic status, 
school size has no effect on higher order skills in either science or mathematics. 
The correlations hover about zero. Similarly, there is no association between a 
school's rurality and its students scores on test of higher order thinking skills. 

(TABLE 1 HERE) 

There are a few other points worth noting ir> Table 1. Empirically, there is 
little to distinguish the measures of science and mathematics achievement. The 
correlation of .87 suggests that the tests developed by NAEP are measuring the 
same construct. Notice also that the nimiber of advanced courses a school offers 
in either science or mathematics is unrelated to its students' higher order skills in 
those subjects. However, our measure of students' willingness to enroll in 
advanced courses does seem to be related to higher order skills in both 
mathematics and science. The higher the average nimiber of semesters of 



ERIC 



16 



15 

advanced courses taken by students, the higher the school mean on the tests of 
higher order skills. 

SES has a significant zero-order relation with both measures of higher 
order skills. The magnitude of its correlation with these variables is typical of its 
relation with more usual measures of achievement. School socioeconomic status is 
also related positively to school size and negatively to location in a rural area. 
Further, it is significantly correlated with the number of advanced science and 
mathematics courses these schools offer. Finally, students attending higher SES 
schools tend to take more advanced mathematics courses than their peers in lower 
status institutions, though this may not be true of science. 

On average, these schools offer about 6 advanced math courses, one more 
than the number of advanced science courses they offer. Students are also more 
likely to take advanced math courses than they are science, though in neither case 
does course taking seem high. That is, in the typical school students take, on 
average, 1.6 semesters of advanced math and 1.1 of advanced science. Notice that 
both school size and rurality are related to course o£ferings in the predicted 
direction for both mathematics and science, and the correlations are sigm'ficant in 
three of the four cases. Large high schools offer more advanced mathematics and 
science coixrses than do small ones, while the curriculum of rural schools is more 
restricted than urban and suburban ones, at least in the case of math. Rural 
students also appear to take fewer advanced mathematics courses than do their 
urban contemporaries. 
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Table 2 reports the results of our regression analyses. We first ran these 
regressions with a product term for a SIZE x SES interaction in the equation to 
test for the possibility that the any relation between school size and higher order 
skills is dependent upon the SES leve' of the school. This interaction term was 
not significant in any regression, and Tisble 2 reports our results after we dropped 
it from the analysis. (We also tested for ti:\e possibility of a RURAL x SES 
interaction and found none.) We ran the a i wc of regressions for each dependent 
variable. First we entered school size, ruralit> and SES in the equation (colimxns 
1 and 3 of the Table), and then added our measu res of course offerings in math 
and science {CRSOFF) and of student course-takin> : in those subjects (CRSTAKE), 

Table 2 Here 

Table 2 makes it abxmdantly dear that school si. o is unrelated to students* 
performance on the LSAY measures of higher order cogm ive skills. The only 
variable significantly related to either outcome is school socioeconomic status. 
Student course taking, significant at the zero-order level, is not significant here, 
undoubtedly because of the intercorrelations among course taking, course offerings 
and school ses. Thus, our hypothesis that the benefits of larger school size accrue 
in the form of students' acquisition of higher order cognitive skills in scien^^ and 
mathematics was refuted. 

Discussion 

We began this study by suggesting that a positive relationship between 
school size and student performance has been masked in previous studies by a 
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systematic neglect of precisely those outcomes that large size can be expected to 
affect. Now, having focused on just those outcomes, we find that size has no 
influence. Th\is, this study joins the numerous others that suggest that school 
size and student achievement are largely unrelated. 

It is possible that our choice of a dependent variable was based on a 
mistaken notion. More specifically, it is possible that higher-order thinking skills 
are taught in all mathematics and science courses, regai^dless of their level, and/or 
that students acquire the bulk of these skills early in their study of a subject. If 
this were so, much of the gain associated with course-taking might occur in the 
introductory courses offered in virtually every high school, regardless of size. 

It if. also possible that these skills are not taught in any course. That is, 
perhaps these capacities are either largely innate or formed early in life as a 
consequence of child raising practices in the home. The vexy high zero-order 
correlation (.87) between the tests of math and science higher order skills is 
consonant with this suggestion. If either of these conditions obtained, the skills 
we have studied would be imafiected by instruction, and hence imafiected by 
variations in course taking patterns between small and large schools. 

There is also a possibility that higher order skills are more a product of 
pedagogical strategies than either course content or the number of courses that 
students take. That is, perhaps these sHUs are developed as a consequence of 
teachers who deliberately pose questions that require those skills, who probe 
students' answers, and who make assignments that force students to think deeply 



18 

about what they know and how it applies to real problems. If such strategies are 
the primary means of attaining higher order thinking skills, then those skills can 
be acquired in any classroom taught by a teacher with the required talents. 

It would be possible to continue spinning explanations for our non-findings. 
The fact remains that this study joins the host of others which suggests little 
advantage to large schools, at least insofar as student achievement is concerned. 
More to the point, the apparent anomaly with which we began this paper remains: 
While large schools offer more advanced courses than do small ones, those 
offerings appear to have no influence on student achievement. 
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TABLE 1 

CORRELATIONS, MEANS. AND STANDARD DEVIATIONS 

N=47 
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One-tail Significance: 
** p < .01 
* p < .05 
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TABLF. - 
REGRESSIOK UBSULTS 



MAIH SCIENCE 

12 3 4 

SIZE -.008 -.007 -.003 -.001 

RURAL 2.375 2.682 2.585 2.493 

SES 9.180** 7.221* 7.252** 7.151** 
CRSOFF -.044 ..519 

CRSTAKE 3.207 3.026 

Intercept 41.41 41.15 43.33 41.91 

R^ .264 .299 .213 .259 



